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(57) Abstract: The present invention is concerned witli rapidly clianging requirements for 
graphical user interfaces. The range of mobile devices and the rate of change in content 
of information accessible therefrom are increasing- One of the drawbacks of having such a 
range of devices is that there is a corresponding range of configurations and capabilities, and, 
in general, each device requires a different Graphical User Interface (GUI), For companies 
developing products for such a range of devices, the infrastructure and API requirements can 
be highly disparate and the development time can be significant. To alleviate these problems, 
an embodiment of the invention provides a means of specifying the requirements of a GUI 
in a single specification, which includes generic descriptions of GUI elements and can be 
interpreted in respect of any type of device. The embodiment has access to device-specific 
GUI items, so that, once the device type and a generic GUI element has been identified, the 
device specific GUI item coixesponding tlieiTeto is readily identifiable. Preferably the speci- 
fication is written as an XML file, which means that it can be stored on a centralised server, 
and is thus accessible by well-known web server access techniques. In one embodiment the 
specification is processed, and GUI created, firom the server computer 301, which then sends 
the created GUI to an appropriate device for display thereon. 
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GRAPHICAL USER INTERFACE 

The present invention relates to graphical user interfaces, and is suitable 
particularly, but not exclusively, for network-connected devices. 

5 

The demand for network-connected devices is growing at a significant rate. In 
response to this demand, new classes of devices such as smart cellular telephones, 
pagers. Personal Digital Assistants (PDA, which includes any small mobile hand-held 
device that provides computing and information storage and retrieval capabilities for 

10 personal or business use) etc. are continually being developed. In addition, traditional 
consumer electronics such as televisions. Digital Versatile Disc and Compact Disc 
(DVD and CD) players are equipped with new capabilities. 

In order to develop applications on any one of these ever increasing device 
types. Sun™ Microsystems has developed a new Java™ application environment, 

15 called "Java 2 Platform, Micro Edition Solution" (J2ME™), which comprises bespoke 
configurations and profiles for different lightweight device types. Among other 
things, these configurations and profiles can be used to create a Graphical User 
Interface (GUI) on a device. 

A configuration is comprised of a virtual machine, which, as is known by those 

20 skilled in the art, enables Java™ applications to run on the device (for further details 
of Java virtual machine, and how it interoperates with a microprocessor running on a 
device, the reader is referred to "The Java™ Virtual machine specification". Sun 
Microsystems Chapter 1.2, Lindholm, T., Yellin, F. 1999. In addition a configuration 
is comprised of core libraries, classes and application programming interfaces (APIs). 

25 There are currently (January 2002) two configurations: the co-called Connected 
Limited Device Configuration (CLDC) and the Connected Device Configuration (CDC). 
The former configuration type, CLDC, is suitable for devices with constrained 
processing capabilities and memory resources, while the latter configuration type, 
CDC, is suitable for devices with more resources. 

30 A profile is a specification that defines device specific application programming 

interfaces (APIs), such as device display features. An example of the relationship 
between configurations and profiles is shown in Figure 1 , which shows two types of 
profile types: MIDP 101 and Personal 103. MIDP 101 is a profile type that is suitable 



wo 03/067427 



PCT/GB03/00518 



2 

for wireless and mobile devices, and links into the CLDC 105 configuration, while 
the Personal profile 103, together with the Foundation profile 106, is suitable for 
devices with more computational power, and co-operates with the CDC 107 
configuration. As can be seen from Figure 1, each configuration CLDC 105, CDC 
5 1 07 interoperates with a Java virtual machine KVM 1 09, CVM 1 1 1 respectively. 

Clearly with this approach, Java™ technology is moving away from the "Write 
Once, Run Anywhere" paradigm, and developers are once again forced to write 
different applications for different platforms. 

10 According to a first aspect of the present invention there is provided apparatus for 
creating a Graphical User Interface for a device, comprising 

a store arranged to store a plurality of processable procedures, each being 
associated with one or more device types; 

receiving means arranged to receive a device independent Graphical User 
15 Interface specification and data identifying the device; 

processing means arranged to process the received Graphical User Interface 
specification in accordance with the received identifying data, so as to identify, from 
the store, a plurality of procedures corresponding to the identified device; and 

combining means arranged to combine the identified procedures into a program 
20 for processing by the device, thereby creating the Graphical User Interface. 

Conveniently the device-independent graphical user interface specification 
identifies display requirements of a graphical user interface. Embodiments of the 
invention identify a device type, and, depending on the device type, interpret the 
25 specification in accordance with device-dependent display capabilities. These device- 
dependent capabilities can be derived from software that has been written for a 
particular device. 

Advantageously at least some of the plurality of procedures are written in 
accordance with object-oriented programming, so that at least some procedures 
30 have, associated therewith, a class, which is associated with a device type. The 
apparatus further includes means arranged to instantiate an object of the identified 
device type and to pass the instantiated object to the processing means, the 
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processing means thereby having access to the procedures associated with the 
instantiated object. 

The processing means is then arranged to receive a reference to procedures 
corresponding to items in the received Graphical User Interface specification 
5 Preferably the GUI specification is written in the Extensible Mark-up Language 

(XML). Alternatively the GUI specification could be written in User Interface Markup 
Language (UIML), which is an XML language specifically for defining user interfaces. 
As a further alternative the GUI specification could be written using a proprietary 
descriptive language having a bespoke standard for the encoding of items such as 
10 GUI widgets, symbols and layouts etc. 

Conveniently, for each item in the GUI specification, embodiments of the 
invention have access to a repository of functions for each device type, so that if, 
for example, the XML specification includes the text < display >, and the device type 
is identified as being a wireless device, an object corresponding to a wireless device 
15 is created and a function of that object, which has the capability of creating a 
display on a wireless device, is then invoked. 

Preferably the processable procedures include procedures for processing one or 
more events in accordance with user input received via the application program. 
Conveniently the apparatus includes means to select one or more of the said 
20 procedures for processing one or more events on the device in accordance with 
items in the Graphical User Interface specification. 

According to the invention there is also provided a method corresponding to 
the apparatus described herein. 

The apparatus can either be run on a server apparatus, in which case the 
25 Graphical User Interface, once created, is transmitted to a device for which it is 
intended. Alternatively the apparatus can be run directly on the device itself. 

The library functions provided by "Abstract Factory™" can be used to invoke 
display components as a function of device type (described by Gamma, E., Helm, R., 
30 Johnson, R., and Vlissides, J. in "Design Patterns: Elements of Reusable Object- 
Oriented Software", Addison-Wesley, Reading, MA, at pp. 87 - 95). These library 
functions are written in accordance with the object-oriented programming paradigm, 
and provide polymorphism of objects. Polymorphism is a concept whereby the 
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functionality of an object is defined by its run-time type. Polynnorphisnn is 
implemented via an interface comprising one base class that contains all methods 
common to different behaviours. The common methods are defined, in the base 
class, as being virtual. The different behaviour is provided by different child classes 
5 (derived from the base class, also known as a sub-class), which implement the 
methods that were defined as virtual in the base class. 

An array can then be created, whose elements are declared as instances of the 
base class; at run-time (i.e. when the elements are instantiated), any sub-class of the 
known base class can replace an element of the array, thereby dynamically defining 

10 the behaviour of the object. 

This functionality can be described by an example: consider a program that 
draws shapes on the screen. First, we declare a base class. Figure, having one or 
more virtual methods therein, which are declared as "abstract". In this example, 
base class Figure includes virtual method Draw. 

1 5 Abstract methods exist only to provide a gateway to multiple different forms of 

a method, and are thus designed to be overridden by a child class. As a result the 
Figure class (base class) does not include an implementation of the Draw method (In 
C-h -h, the Draw method is called pure virtual member function, in Java the Draw 
method is simply called an abstract method). 

20 Continuing with the example, next we define child classes: Square and 

Triangle, which implement the method Draw: 

II Common for Square and Triangle. 

Polygon = class (Figure) 

private 

25 a: integer; // Length of the side, 
end; 

Square = class { Polygon) 

public 

30 procedure Draw/ override; // Overrides Flguxe .Dr3LVj . 

end; 

Triangle = class { Polygon) 
public 

35 procedure Draw; override; // Overrides Figure ,Dra^f/ . 
end; 

// Include some implementation for Sqfuare . Draw, Triangle. Draw (not shown) 

Next we create a data structure. Figures array: 
40 Figures = array[0..2] of Figure; 
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which is actually an array of pointers declared to point to objects of Figure class. 
This means that, at run-time, we can place children of the Figure class in the 
array, i.e.: 

var 

5 F: Figures ; 
i: integer; 

F[0] = Square . Create ; 
F [ 1 ] = Triangle . Create ; 

10 // Draw figures. 

//We don't need to know the actual class o£ these objects I 

for i:=Low(F) to High(F) do 
F [ i ] . Draw 

15 The Abstract Factory™ libraries include abstract base classes and child classes 

that provide polymorphism. Referring to Figure 2, instead of hard-coding display 
widgets into an application A (e.g. an application called "GUI creation"), there is an 
interface 201, which is representative of a virtual method within a base class, and 
which can be passed as a parameter to the Application A. When application A is run, 

20 the interface 201 is cast to whichever object corresponds to the client type, and 
sent as a parameter to the application A. Thus, for example if a client C is running 
Motif, then when the Application A requests 212 a scrollbar, the interface 201 will 
automatically pass, to the Application A, a reference 214 to a MotifScrollBar object. 
An Open Source product called "Abstract User Interface Toolkit" (AUT) also 

25 makes use of the Abstract Factory design approach. GUI requests are written in an 
XML specification, and the client translates the specification in order to identify the 
object to be requested. Display components corresponding to the identified object 
are passed to the display application via the Interface 201 as described above. This 
Open Source product has been written for "real" windows (in this case Motif, 

30 Macintosh OS X and Win32) that run on a single device (conventional PCs). Each 
window type relates to a different operating system (Unix, Mac and Windows 
respectively) and has a separate XML specification corresponding thereto. By 
comparison, in embodiments of the invention a single GUI specification is written 
and this can be used for any device type. 

35 Essentially the "Abstract User Interface Toolkit" is designed for conventional 

desktop machines, where the actual windowing functionality is identical, but the 
means of invoking the functionality is not. By comparison, embodiments of the 
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invention are concerned with devices where the graphical interface functionality 
varies considerably between device types: cell phones and pagers, for example, have 
snnall screens, limited battery life and low memory resources; whereas entertainment 
environments offer high speed connectivity to a network and have around 1 6 MB 
5 memory. 

Further features and advantages of the invention will be apparent from the 
following description of preferred embodiments of the invention, which are given by 
way of example only and with reference to the accompanying drawings, in which 
10 Figure 1 shows an overview of the "Java 2 Platform, Micro Edition Solution" 
(J2ME™) application environment with which embodiments of the invention 
interoperate; 

Figure 2 is a schematic representation of "Abstract Factory™" design utilised by 
embodiments of the invention; 
1 5 Figure 3 is a schematic representation of the environment in which embodiments of 
the invention operate; 

Figure 4 is a schematic representation of an embodiment of the invention generally 
referred to as GUI engine; 

Figure 5 is a flow diagram showing steps involved in a first embodiment of the 
20 invention; 

Figure 6 is a schematic representation of an object created by the parsing program 
forming part of the GUI engine shown in Figure 4; 

Figure 7 is a schematic representation of the functionality of the interface forming 
part of the GUI engine of Figure 4; 
25 Figure 8 is a flow diagram showing steps involved in processing the object shown in 
Figure 6; 

Figures 9a and 9b are illustrations of a GUI created by the GUI engine shown in 
Figure 4; and 

Figure 10 is a schematic representation of processing of an event listening program. 

30 

OVERVIEW OF AN EMBODIMENT OF THE INVENTION 

Referring to Figure 3, an embodiment of the invention runs on a server 
computer 301 , which is in communication with a range of devices 303a 303b, 303c 
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via a public land mobile network (PLMN) (e.g. a GSM - compatible digital cellular 
network) N1 or, if the devices 303a, 303b, 303c are equipped with Wireless Ethernet 
network (WLAN) technology, via a 802.1 1 -compatible Wireless Ethernet network 
(WLAN) N2 (or any other short range network). Additionally, or alternatively the 
5 devices 303a, 303b, 303c could communicate with the server 301 via GPRS and/or 
UMTS, which are mobile networks capable of sending and receiving packet switched 
data. 

The PLMN network N1 can be connected via a gateway G1 to a public switched 
network (PSTN) N3. Figure 3 shows 3 device types only, but many more are 
10 possible. 

The PSTN N3 is interconnected with an integrated services digital network (ISDN) 
N4 via a gateway G2 (e.g. a local or international switching centre), and is connected 
via an ISDN line LI to the server computer 301, and to local area network (LAN) N5. 
One or more database servers 01 are accessible by the server computer 301 via the 
1 5 local area network N5. In addition the ISDN network N4 is connected to the WWW via 
line L2. 

Information sources, held for example on servers IS1-IS4, are distributed 
throughout the networks N1-N6. Only four servers are shown, however other 
distributions are envisaged. A user operating a smart phone device 303a may wish 

20 to receive information from any or all the information sources in the system. 

With the proliferation and continuing development of mobile devices 303a - 
303c, and the rate of change to the type and content of information accessible 
therefrom, it is vital that the user interface can be easily modified. However, one of 
the drawbacks of having such a range of devices is that there is a corresponding 

25 range of configurations and capabilities, meaning that each device requires a 
different GUI. For companies developing products for such a range of devices, the 
infrastructure and API requirements can be highly disparate and the development 
time can be significant. 

To alleviate these problems, an embodiment of the invention provides a means 

30 of specifying the requirements of a GUI in a single specification, which includes 
generic descriptions of GUI elements and can be interpreted in respect of any type of 
device. The embodiment has access to device-specific GUI items, so that, once the 
device type and a generic GUI element has been identified, the device specific GUI 
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item corresponding thereto is readily identifiable. Preferably the specification is 
written as an XML file, which means that it can be stored on a centralised server, 
and is thus accessible by well-known web server access techniques. In one 
embodiment the specification is processed, and GUI created, from the server 
5 computer 301, which then sends the created GUI to an appropriate device for 
display thereon. 

A particular feature of the invention is that, because a single GUI specification 
is used, the same information and menu options will be available on all devices used 
by a user (albeit arranged differently). It is well recognised that humans prefer to use 
10 devices with which they feel familiar, because it means that information is delivered 
and/or accessed to and/or by a user in a manner with which they are accustomed. 
Thus this conformance of GUI components across devices is thus a significant 
advantage. 

15 Referring now to Figure 4, the server computer 301 comprises a central 

processing unit (CPU) 401, a memory unit 403, an input/output device 405 for 
connecting the server computer to the devices 303a, 303b, 303c and to the 
database 01; storage 407, and a suite of operating system programs 409, which 
control and co-ordinate low level operation of the server computer 301 . 

20 Generally an embodiment of the invention is referred to as a GUI engine 400, 

and comprises at least some of programs 411, 413, 415, 417. These programs, 
which are described in more detail below, are stored on storage 407 and are 
processable by the CPU 401. The GUI engine 400 also includes, or has access to, a 
database 01, which stores methods corresponding to device types. It is understood 

25 that each of the programs 411, 413, 415, 417 could comprise a suite a computer 
programs, and is preferably written in an object-oriented programming language, 
such as Java™. 

The programs include a program 41 1 for detecting a type of device for which 
the GUI is intended, a program 413 for requesting and/or receiving a specification of 
30 the GUI requirements, a program 415 for parsing the specification, and a program 
417 for processing the parsed specification so as to create a GUI comprising 
features that are appropriate to the detected device type. Conveniently at least some 
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of the programs 411, 413, 415, 417 make use of, and extend, library functions 
supplied by Abstract Factory™ described above. 

The specification of GUI requirements can be written using the Extensible 
Mark-up Language (XML). This specification is device-independent, so that, for a 
5 particular GUI format, a single specification applies for all of the devices 303a, 303b, 
303c. Essentially, once the detecting program 41 1 has detected a device type, it 
creates an object of that device type. When the processing program 41 7 processes 
the specification, the methods and features of the created object are automatically 
run and retrieved, thereby creating a GUI that has been customised for the detected 
1 0 device type. 

Referring to Figures 5, 6, 7 and 8, an embodiment of the invention will now be 
described in more detail. Figure 5 is a flow diagram showing steps carried out by the 
GUI engine 400 when creating a GUI for a smart phone 303a, and Figure 6 is a 

15 schematic representation of a device-independent GUI specification. Figure 7 is a 
schematic block diagram showing device-dependent objects and methods thereof 
that are stored in the database 01 and accessed by the processing program 417. In 
Figure 7 the complete lines with arrows indicate process steps, and the dash-dotted 
lines indicate access relationships. Figure 8 is a flow diagram showing steps carried 

20 out by the processing program 417. 

In the description of the current embodiment, it is assumed that the programs 
411, 413, 415, 417 operate in "object-oriented" fashion; that is to say that the data 
associated with the programs is "encapsulated" so as to be accessible only by 
associated control programs, acting in response to "messages" (which need not, 

25 however, be physically transmitted but could simply be data passed via the stack of a 
single computer). 

Referring first to Figure 5, the GUI engine 400 receives 501 a request for a GUI 
from the smart phone 303a, and sends this request to the detecting program 411, 
which is activated to identify the type of device that has sent the request. When the 
30 GUIs are created using Java™, they are run on a Java virtual machine running on the 
device 303a. The type of Java virtual machine will vary from device to device, 
depending on the configuration 105, 107 running thereon, so identifying the type of 
Java virtual machine provides a means of detecting the type of device. 



wo 03/067427 



PCT/GB03/00518 



10 

Accordingly, at step 503 the detecting program 41 1 sends a request for the 
type of virtual machine running on the device 303a. This involves remotely invoking 
a system method, getProperty 0 on the device, via Remote Method Invocation (RMI). 
RMI is part of the Java™ programming language library functions and involves a local 
5 surrogate (stub) object (not shown) on the server machine 301 managing invocation 
on a remote object (device 303a). The system method getProperty 0 is available 
from the configuration API running on the requesting device 303a. The following are 
examples of the request messages invoked on the device at step 503: 
J2SE 

10 To check if the device is running J2SE the detecting program 411 asks if the 

system property java. runtime. name contains the value Java(TM) 2 Runtime 
Environment. If the string is anything else or null then the platform is not J2SE. 
Examples of devices that run J2SE are laptops and desktop PCs. 

e.g. 

15 public static final boolean isJ2SE; 

String property = System. getProperty ( "java. runtime .name") ; 
if (property != null) 

isJ2SE = (property . indexOf ( "Java (TM) 2 Runtime Environment") != -1) ; 
20 else 

isJ2SE = false; 

Persona! Java 

To check if the device is running pJava the detecting program 41 1 asks if the 
25 system property os.name contains the value windows CE. As Windows CE only 
runs Personal Java the platform will be pJava (pJava devices include the iPAQ (high- 
spec PDA)). If the string is anything else or null then the platform is not pJava. 

e.g. 

public static final boolean isPjava; 

30 

property = System . getProperty ( "os .name" ) ; 
if (property != null) 

isPjava = (property . indexOf ( "Windows CE") != -1) ; 
else 

35 isPjava = falser- 
It should be noted that during 2002 Sun™ Microsystems will replace PersonalJava™ 
with the Configuration/Profile architecture, (CDC/Foundation Profile/Personal Profile) 
shown in Figure 1 . As a result the above code will be modified so that, when sent a 

40 getPropertyO request, the System returns an identifier indicative of a device 
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operating in accordance with a Config/Profile. The skilled person would readily 
appreciate how to make such a modification. 

MIDP 

5 To check if the device is running MIDP the detecting program 41 1 asks if the 

microedit ion. prof iles system property contains the string MIDP. If the string 
is anything other than MIDP or null then the platform is not MIDP. Examples of 
devices that run MIDP are smart phones and Palm Pilots. 

e.g. 

10 public static final boolean isMIDP; 

property = System. getProperty { "microedition. prof iles" ) ; 
if ( profiles != null ) 

isMIDP = { property. indexOf ("MIDP-") 1« -1) ; 
1 5 else 

isMIDP = false; 

By invoking a series of getProperty 0 messages the detecting program 411 
identifies 505 which platform, and thus which type of device, has sent the request 
20 message. 

The detecting program 41 1 then creates 507 an object OD (not shown) of this 
identified device type (equivalent to square . create in the example given in the 
introduction section). Each device object has methods and variables corresponding 
thereto, so that, by creating an object of the detected device type, these associated 

25 methods and variables can thereafter be invoked. 

The GUI engine 400 then activates the receiving program 413 to retrieve a 
specification of the GUI requirements. When the specification is written in XML, it 
can be accessed from a web server, via a Uniform Resource Locator (URL) submitted 
with the request sent at step 501 . If the request (step 501) does not include a URL, 

30 or some indication of an access point for the specification, the receiving program 
413 can access a default GUI specification (or a default URL) from the GUI engine 
400. Accordingly, assuming a URL is submitted together with the request at step 
501 , the receiving program 41 3 sends 509 a request for an XML file to a web server 
at the submitted URL, and receives 51 1 an XML specification therefrom. 

35 The parsing program 415 is then activated to parse the XML specification. In 

one arrangement the parsing program 415 creates and stores 513 the specification 
as an XML object 600 comprising a series of nodes, each of which corresponds to a 
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tag in the XML specification, and is readily processable by the processing program 
417. An overview of XML and tags is given on pages 2 - 5 of McLaughlin, B., Java 
and XML, O'Rielly, 2000. (ISBN 0-596-00016-2), and an overview of parsing of 
XML is given on page 23, and 46-57. 
5 An example of such an XML specification is given below (where a tag is given 

by :<tag>), and a parsed XML object corresponding thereto is schematically shown 
in Figure 6: 

<display> 601 // display tag : "display" root element 

<panel> 6 03 // panel tag: "external" panel 

10 <position></position> 605 

<name>3rd form</name> 607 
< layout > 609 
<name>BorderLayout</name> 611 
</ layout > 

15 <panel> 615 // inner panel 

<position>CENTER</position> 617 
<name>lst f orm</name> 619 
< layout > </ layout > 621 
</panel> 

20 <choicegroup> 633 

<position></position> 635 
<name>This is Choice Group</name> 637 
<type>MULTIPLE</type> 639 
<chQice>A</choice> 641 
25 <choice>B</choice> 643 

<choice>C</choice> 645 

</choicegroup> 

</display> 

30 The XML object 600 created at step 513 is then processed, step 515, by the 

processing program 417. This involves processing each node of the XML object 600, 
as described In detail with respect to Figure 8 below. 

As stated above, at least some of the programs 411 - 417 make use of the 
Abstract Factory™ libraries (described above with reference to Figure 2). Referring to 

35 Figure 7, the GUI engine 400 has an interface 701, which is arranged to receive a 
device-independent request for some aspect of a GUI from the processing program 
417, and, depending on the object OD created at step 507, the interface 701 will 
refer to a device-dependent method relating to the request. These methods, which 
are described in more detail below, can be stored in the database 01 and are 

40 automatically accessible by the object when it is created at step 507. 

Figure 7 illustrates the functionality of the interface 701 for the first node 
< display > of the example XML specification given above. Referring to Figure 7 the 
processing program 417 sends S 7.1 a request for a "display" to interface 701. The 
interface 701 returns a reference to a display function S 7.2 of the object created at 
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step 507 back to the processing program 417 (essentially OD.createDisplayO (cf 
F[i] .draw = square . create . draw ( ) in the example given in the introduction)). 
This means that, at runtime, the specific implementation of the createDisplayO 
method corresponding to object OD will be invoked. 
5 Prior to sending the request at step S 7.1, the processing program 417 has to 

identify which generic GUI element to request. For example, this can involve 
accessing a Hashtable data structure, which stores relationships between XML tags 
and generic GUI elements. A generic GUI element corresponding to an XML tag 
<name> can be identified by invoking a method to retrieve a node corresponding to 
10 <name> from the Hashtable, in accordance, for example, with the following 
psuedo-code: 

Hashtable component Mapping ; 
Go to first node of XML 
Generic Component gc ; 
15 Do (whilst not at end of XML structure) 

gc= (GenericComponent) coraponentMapping . get (elementName) ; 

gc . create ( ) ; 

get next element of node 
end do 

20 Alternatively this can involve processing a sequence of "if ... then" statements, 

where the "if" is conditional on the type of node (in the code fragment set out 

below, the type of node is object elementName): 

if (elementName .equals ( "display" ) ) 

{ 

25 // create display. . . 

f actory .createDi splay (app) ; 

number = elementRoot . getChildCount ( ) ; 
// System. out .print In { "\n"+number) ; 
30 if (number==0 ) return; 

for(int i=0; i<number; i++) 

{ 

if (elementRoot -get Type (i) ==Xml . ELEMENT) 

35 element = elementRoot . getElement (i) ; 

processGUlSpec (element , null) ; 

} 

} 

} 

40 This alternative example includes several object instantiations equivalent to 

Square . create ( ) that was discussed in the introduction. 



wo 03/067427 



PCT/GB03/00518 



14 

Steps S 7.1 and S 7.2 are repeated for each node in a depth first with back 
tracking manner {i.e. top to bottom, left to right - that is, the first route through the 
XML object 600 is the left-most nodes, and when a leaf node is encountered, the 
route is back tracked until an adjacent right hand node is encountered whereupon 
5 the route burrows down this adjacent node until a next leaf node is encountered. 
This process is repeated for all nodes in the XML object 600). Tree structures (such 
as is shown in Figure 6) and depth first traversal are described in Budd, T. 2001, 
Ciassic Data Structures m Java, Addison Wesley Longman, Oregon, at, respectively 
pp 325-330, and pp 343-344. 

10 This method is illustrated in Figure 8, for the example XML specification shown 

in Figure 6. The processing program 417 reads in 801 the first node in the XML 
object 600 and checks 803 whether the node is of "display" type. If it is (as it is in 
the current example), the processing program 417 sends S 7.1 a request for a 
"display" (via the interface method createDisplay () ) to object CD (which sub- 

15 classes interface 701) created at step 507, and returns S 7.2 the reference back to 
the processing program 417, which stores 805 this as a reference to a new display 
area. This means that, at runtime, the specific implementation of the createDisplayO 
method corresponding to object CD will be invoked. 

The processing program 417 then reads in 807 the left-most node 603 and 

20 repeats step 803 in respect thereof, moving onto step 809 because the next node is 
not a "display" node. At step 809 the processing program 417 checks whether the 
node is of "panel" type. As in this example it is, the processing program 417 sends 
S 7.1 a request for a "panel" (via the interface method createPanel () ) to object 
OD created at step 507, and returns S 7.2 the reference back to the processing 

25 program 417. This means that, at runtime, the specific implementation of the 
createPanelO method corresponding to object OD will be invoked. 

The processing program 417 then adds 811 the reference to the panel to the 
reference stored at step 805. The processing program 417 then checks 813 whether 
this panel is the first panel node encountered in the XML object 600, and if it is, it 

30 sets 815 this panel as the root display element. If this panel is not the first panel 
node, step 815 is omitted, and the processing program 417 skips to step 817, 
where the processing program 417 reads in 817 the left-most node 605 and repeats 
steps 803, 809 in respect thereof. 
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In the current example this node Is neither a "display" nor a "panel" node, so 
the processing program 417 omits steps 803, 809, and extracts 819 whatever 
information pertains to the node (for node 605 there is no additional information to 
be extracted). 

5 Steps 819 - 823 are best illustrated for the case of node 633. When the 

processing program 417 reaches node 633, it sends a request for a "choice group" 
(via the interface method createChoiceGroup () ) to object OD created at step 
507, and returns S 7.2 the reference back to the processing program 417. The 
processing program 417 then adds 823 the reference to the panel set as root display 

10 element at step 815. 

Whenever the processing program 417 comes across a terminal branch node 
(e.g. nodes 605, 607a, 609a) it backtracks 880 through the XML object 600 to 
identify a next untested node (e.g. having reached node 605, the processing 
program 417 backtracks to node 603). 

15 Returning to Figure 5, once the processing program 417 has processed all of 

the nodes in the XML object 600, the processing program 417 can be said to have 
created a bespoke GUI class, and, at step 517, the class is sent to the requesting 
device (here smart phone 303a) as byte code. The client C, which is running on the 
device 303a and is in operative association with a Java Virtual Machine, then reads 

20 in the byte-code and converts it into a class. Thereafter the client C instantiates a 
new instance of the class - namely a GUI object - essentially displaying 519 the GUI 
on the device 303a. This effects "run-time" instantiation of the referenced methods, 
as described with reference to the Draw and Figure example in the introduction 
section. 

25 Figures 9a and 9b each show a GUI that has been created from the same XML 

specification, for a smart phone and a PDA respectively. The two interfaces show 
identical information: referring to Figure 9a menu options Xmllnfo 901, Alert 903, 
TxtBox 905 TextArea 907 correspond to Xmllnfo 911, Alert 913, TxtBox 915 
TextBox 917 on Figure 9b. 

30 

Second Embodiment 

A second embodiment, generally similar to that described in Figures 1 - 9, will 
now be described, in which like parts have like reference numerals and will not be 
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described further in detail. In the first embodiment, one or more server computers 
301 receive requests for GUIs from devices. In a second embodiment, each device 
303a, 303b, 303c has a GUI engine 400 running thereon, so that a request for a 
GUI is processed locally on the device itself. 



Referring to Figure 10, in order for the device 303a to process user input 
received via the GUI loaded at step 519, it needs to run event listening code 1001. 
As is known in the art, event listening programs listen for inputs (e.g. user input via 
a keyboard, or via a keypad) and have procedures that respond, with predefined 
10 actions, to received inputs. 

Typically a developer will write event listening code 1001, which corresponds 
to items in the GUI specification; for each GUI specification, the developer only 
needs to write one version of event-listening code, which can thereafter be sent to 
any device. 

15 The event listening code 1001 can be stored on the server computer 301 (e.g. 

as a library class). When the event listening code 1001 is written in the Java™ 
programming language, it is compiled, on the server computer 301, into byte-code, 
sent to the device 303a, and received by a program 1003 running on the device 
303a. As stated above, the client C running on the device 303a is in operative 

20 association with a Java Virtual Machine, which is arranged to interpret the byte-code 
of the program 1001 (once translated it into a class file). 

The following code fragment is an example of event listening code for handling 
menu selection (in the present embodiment the event-listening code 1001 is a class 
that is written using the Abstract Factory™ libraries): 

25 // create menulistener for application's menu 

menulistener = factory . menuListener (new LEAPActionO 



5 



30 



public void action(){ 
try{ 

//get the menu option selected by the user 

int i = factory .getMenuCommand () ; 
switch ( i) 



35 



case 0 : 



40 



//when 'Xmllnfo' menuitem is pressed, info defined in xml 
//becomes visible 

System. out .print In ( "Xmllnfo pressed") ; 
factory . setDisplay (components .get ( "Xmllnfo" ) ) ; 
break; 



case 1 : 
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//when 'Alert' menuitem is pressed. Alert becomes 
// visible 

System. out .print In ("Alert pressed") ; 
factory . setDisplay (components. get ( "XmlAlert" ) ) ; 
5 break ; 

case 2 : 

//when 'TxtBox' menuitem is pressed TxtBox 
//is made visible on panel4 . . . 
System . out . println ( " Info pressed" ) ; 
10 factory . addToPanel {components .get ( "4th form")/ 

factory. textArea( "TextBox. . . " , "Hello! ! ! " ,100) , "NORTH") ; 

break ; 
case 3 : 

//when 'Panels' menuitem is pressed panels 
15 //is made visible... 

System. out .println ( "Panels pressed") ; 
factory . setDisplay (components . get { " 3rd form" ) ) ; 
break ; 
case 4 : 

20 //when 'Help' menuitem is pressed info box 

//is made visible. . . 

factory. setDisplay {factory. in£oScreen(" Sorry, no help! !!")); 
System. out .println { "Help pressed") ; 
break ; 
25 default: 
break ; 

} 

} catch (Exception e) { 

e . pr intStackTrace ( ) ,- 

30 } 

The client C firstly instantiates an object of the device (it can reuse the device 
object DO created at step 507). Once the JVM has interpreted the received byte- 
code into an event listening class, the client C instantiates the said class, thereby 
realising references to certain event listening methods (in the example above, 
35 menuListenerO and getMenuCommandO) via the interface 701. This essentially 
involves requesting a reference of a menuListener, or getlVlenuCommand etc. 
function of the object DO (i.e. calling "OD. menuListener"). 

The client C can then invoke the method(s) using the reference{s) that is/are 
received from the interface 701 at run-time. Considering the example of a pJava 
40 device, a reference to the method menuListenerO for this device type effectively 
activates method ActionListener () at run-time. 

The reference to menuListenerO is then coupled to components of the GUI, 
thereby "attaching" the event listeners to the GUI (referring to the code fragment 
below): 

45 // add menulistener to the application's menu 
try{ 

factory. setMenuListener (components .get ( "Menu" ) , menulistener) ; 
50 } catch (Exception e) { 
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e .printStackTrace ( ) ; 

} 

// components.getC'Menu") identifies which GUI component to attach the listener 
5 // to (here a menu connponent), and nnenuListener is effectively a reference to 
// "OD.menuLlstenerO" 

Mapping between generic GUI eiements and device specific GUI items 

As stated above, an interface 701 essentially provides a means of 

10 implementing mappings between generic GUI elements and device specific GUI items 
(returning to Figure 7, "Display" is a description of a generic GUI element, and 
"MIDIet display" is a GUI display item specific to the smart phone 303a). In the first 
and second embodiments the actual implementation of the mappings is via 
polymorphism, as each type of device is represented by a sub-class, each of which 

15 conform to methods corresponding to their base class, interface 701. In another 
embodiment the mappings are implemented as a look-up table between generic 
description and a specific item; this is described in more detail later. 

Returning to the first and second embodiments, each specific GUI item involves 
one or more classes and methods. As will be appreciated, identifying GUI items for 

20 some generic GUI elements is non-trivial, not least because the GUI architecture for 
different Java™ platforms can vary considerably in terms of the relationships 
between GUI items, the hierarchy of GUI items and the inadequate, and in some 
instances excessive, equivalencies of GUI objects between platforms. 

The following table presents examples of GUI items for MIDP and pJava 

25 devices corresponding to generic GUI descriptions. At least some of the items 
involve bespoke classes and methods (indicated by "bespoke"), while others are 
provided by the API corresponding to the device (indicated by "API class"). This list 
is not exhaustive. 



Description 
of Generic 
GUI element 


MIDP GUI item 


PJava GUI 
item 


Comments 


Display 

(main 

window) 


Display 
(API class) 


Frame 
(API class) 
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Panel 

(screen main 

within 

window) 


Form 

(API class + 
bespoke 

tweaking of 
methods therein) 


Panel 

(API class) 


ranei components can ue aaaea 
using an appropriate layout 
whereas Form has no layout - it 
puts all components under one 
anotner. in aaaiiion r^anei can 
contain other Panels whereas 
Form cannot contain other forms. 
These difficulties were solved 
with a restriction on buttons. 


Textfield 
(allows user 
to enter 
text) 


Textfield 
(API class) 


Textfield 
(API class) 




TextArea 


Textfield 
(API Class) 


TextArea 

\Arl Class) 


TextArea in pJava has no 
ecjuivaience in iviiur, so lexirieia 
can be used. As an alternative, 
the TextBox class could be used 
(although TextBox is a separate 
ocreen cioSo, rainer xncin an iicm 
class so it cannot be added to the 
Form). 

See parts 907, 917 on Figures 


Date 


DateField 
(API class) 


MyDateField 
(bespoke) 


pJava classes do not include date 
and time fields, so MyDateField 
was created based on Abstract 
Windows Toolkit™ (AWT) classes 


Time 


TimeField 
(API class) 


iviy 1 imerieiu 
(bespoke) 


pjava Classes ao noi inciuoe uaie 
and time fields, so MyDateField 
was created based on AWT 
classes 


ChoiceGroup 
(allows user 
to select 
from some 
options) 


v^noicevjfoup 
(API class) 


^neoKDQx 
(API class) 




AlertScreen 
(displays 
data on the 
screen) 


Alert 

(API class) 


LtAr Alert 
(bespoke) 


LtArAieri Class exienas ine 
Dialog class available from the 
pJava API 


ivienu 

(allows user 
to select 
from 
options) 


1 ict /API ol9cc\ 


1 UUIUdl OL 

buttons 
(bespoke) 


Tho m^nii tf^m in n 
1 lit? iiiciiu 1 LCI 1 1 til fj*jovcL 10 

implemented as a toolbar, which 
is placed at the top of the frame 
and includes various buttons 


Button 

(device- 
specific soft 
buttons) 


Command (API 
class) 


Button (API 
class) 
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Listen (listen 


CommandListener 


ActionListener 




for events) 


(API class) 


(API class) 





Whenever new XML tags are added, or a generic GUI element corresponding to 
an XML tag changes, the hashtable (or series of switch statements) described above 
requires updating, so that, when the graphical user interface specification is 
5 processed, the processing program 417 can identify which request to send at step S 
7.1. 



In a third embodiment, polymorphism is not used. This means that the classes 
and methods are explicitly written for each device type, and an object of each type 

10 of device has to be explicitly instantiated when creating the GUI (i.e. no instantiation 
of object at run-time). Accordingly at some point prior to step 801, the processing 
program 417 receives an identifier representative of the said device type and, when 
processing the XML object 600, it directly invokes classes and methods relating 
thereto so as to add a specific component to a specific display that relates to that 

1 5 specific platform of the device type. The following pseudo-java code illustrates how 
this embodiment could operate; 



PjavaDisplayObject p javaDisplay ; //this object only runs in a pjava environment 
MIDPdisplayObject midpDi splay; //this object only runs in a MIDP environment 

20 

If (the system is running on pJava) 

Create a gui based on p j avaDisplay; 
processXMLf orPj ava (p JavaDisplay) ; 
25 Elseif {the system is running MIDP) 

Create a gui based on midpDisplay; 
ProcessXMLf orMIDP (MIDPDisplay) ; 
End if 



30 where ProcessXMLf orPj ava (pJavaDisplay) is defined as: 

void ProcessXMLf orPj ava (PjavaDisplayObj ect pJavaDisplay) { 

Get the GUI element description from the next node; 
35 While (more nodes exist) 

If (node is x) 

Create a pJava GUI element relating to x and add it to the 
pJavaDisplay ; 
40 Elseif (node is y) 

Create a pJava GUI element relating to y and add it to the 
p Ja vaD i sp 1 ay ; 

Elseif (node is z) 
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endif 



5 endwhile 
} 



and ProcessXMLforMiDP (MiDPDisplay) is defined as: 
10 void ProcessXMLf orMIDP (MIDPDisplayObject midpDisplay) { 

Get the GUI element description from the next node; 
While (more nodes exist) 
If (node is x) 

1 5 Create a MIDP GUI element of relating to x and add it to the 

MiDPDisplay; 

Elseif (node is y) 

Create a MIDP GUI element of relating y and add it to the 
MiDPDisplay; 
20 Elseif (node is z) 

endif 
endwhile 

} 

25 

A fourth embodiment similarly does not use polymorphism (interface 701). 
Instead, the mapping between generic GUI description and specific GUI item is 
provided by a simple lookup table of the following form: 



Generic GUI description 
(node) 


Specific GUI item (pJava) 


Specific GUI item (MIDP) 


Display 


Display: invoke method 
CreateDisplay {...} 


Form: invoke method 
CreateForm {...} 



30 



The processing program 417 then operates as follows: 

void ProcessXMLf orMIDP (MIDPDisplayObject midpDisplay) { 

Get the GUI element description from the next node; 
35 While (more nodes exist) 

Select and create a MIDPGUIItem corresponding to this node from the 

mapping 

endwhile 

} 

40 

void ProcessXMLf orpJava (pJavaDisplayObject pJavaDisplay) { 

Get the GUI element description from the next node; 
While (more nodes exist) 
45 Select and create a pJavaGUIItem corresponding to this node from the 

mapping 
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endwhile 

} 

Additional Implementation details: 

The programs 411 - 417 are preferably written using Java™, which means that the 
programs only need to be written once, whereupon they will work on any device 
having a Java Virtual Machine running. The programs could be written using C-h + 
or C# (C-sharp), whereupon they would have to be compiled for each device type (or 
more specifically, for each operating system type/processor type combination). As a 
further alternative the programs could be written using an interpreted language like 
Visual Basic™, but the GUI created therewith could only be run on devices running 
Microsoft™ operating systems, and would have to be compiled specifically for other 
operating systems. 

If the programs were written using the ^programming language, the functionality 
of the interface 701 would be provided by a so-called virtual base class and pure 
virtual member functions (mentioned in the introduction). 
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CLAIMS 

1 . Apparatus for creating a Graphical User Interface for a device, comprising 

a store arranged to store a plurality of processable procedures, each being 
5 associated with one or more device types; 

receiving means arranged to receive a device independent Graphical User 
Interface specification and data identifying the device; 

processing means arranged to process the received Graphical User Interface 
specification in accordance with the received identifying data, so as to identify, from 
10 the store, a plurality of procedures corresponding to the identified device; and 

combining means arranged to combine the identified procedures into a program 
for processing by the device, thereby creating the Graphical User Interface. 

2. Apparatus according to claim 1, wherein the device independent Graphical 
15 User Interface specification comprises a plurality of items arranged hierarchically 

with respect to one another. 

3. Apparatus according to claim 1 or claim 2, wherein at least some of the 
plurality of procedures are written in accordance with object-oriented programming, 

20 so that the at least some procedures have, associated therewith, a class, which is 
associated with a device type. 

4. Apparatus according to claim 3, further including means arranged to 
instantiate an object of the identified device type and to pass the instantiated object 

25 to the processing means, the processing means thereby having access to the 
procedures associated with the instantiated object. 

5. Apparatus according to claim 4, wherein the processing means is arranged 
to receive a reference to procedures corresponding to items in the received Graphical 

30 User Interface specification. 

6. Apparatus according to claim 2, further including a look-up table detailing 
mappings between said items and device-specific procedures, wherein for each of 
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said items in the Graphical User Interface specification, the processing means is 
arranged to select, from the look-up table, procedures that correspond to the said 
item. 

5 7. Apparatus according to any one of the preceding claims, wherein the 

processable procedures include procedures for processing one or more events in 
accordance with user input received via the application program. 

8. Apparatus according to claim 7, including means to select one or more of 
10 the said procedures for processing one or more events on the device in accordance 

with items in the Graphical User Interface specification. 

9. Server apparatus for creating Graphical User Interface comprising 
means arranged to receive a request for a Graphical User Interface from a device; 

1 5 apparatus according to any one of the above claims; and 

transmitting means arranged to transmit, to the requesting device, the program. 

10. Server apparatus according to claim 9 when dependent on claim 8, wherein 
the transmitting means is further arranged to transmit, to the requesting device, the 

20 or each selected procedures. 

11. A device comprising 

apparatus according to any one of claims 1 to 8; 
means arranged to receive the said program, and 
25 means arranged to process the received program, thereby displaying the graphical 
user interface on the device. 

12. A device according to claim 11 further including means arranged to receive 
user input via the received program and to process one or more events on the device 

30 in accordance with the received input. 

13. A method of creating a Graphical User Interface for a device, comprising 
receiving data identifying the device; 
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receiving a device independent Grapliicai User Interface specification; 

accessing a store arranged to store a plurality of processable procedures, eacli 
being associated with one or more device types; 

processing tine received Graphical User Interface specification in accordance 
5 with the received device data so as to identify, from the store, one or more 
procedures corresponding to the received device data; and 

combining the identified procedures into a program for processing by the 
device, thereby creating the Graphical User Interface. 

10 14. A method according to claim 13, in which the Graphical User Interface 
specification comprises a plurality of display items arranged hierarchically with 
respect to one another, and the step of processing the received Graphical User 
Interface specification involves 

reading in a first display item occurring at the top of the hierarchy; 
1 5 selecting a procedure corresponding both to the device type and to the first display 
item; 

recursively identifying items connected to the first display item; and 

for each identified item, selecting a procedure corresponding both to the device type 

and to the identified display item. 

20 

15. Apparatus for creating a program for a device, comprising 

a store arranged to store a plurality of processable procedures, each being 
associated with one or more device types; 

receiving means arranged to receive, or to access, data identifying the device 
25 and a device independent specification; 

processing means arranged to process the received specification in accordance 
with the received identifying data, thereby identifying, from the store, one or more 
procedures corresponding to the identified device; 

combining means arranged to combine the identified procedures into a program 
30 for processing by the device. 
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